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ABSTRACT 

A longitudinal study of achievement data from public 
schools in Durham County, North Carolina, was undertaken to examine 
individual growth curves as well as to chart the progress of 
educational institutions and the effects of aggregation on the study 
of growth. A ttfo*level hierarchical linear model was used in the 
study to specify individual growth and to relate the parameters of 
the growth model to a relevant background variable. Computer analysi 
was conducted using TIMEPATH, a program written in GAUSS, a matrix 
programming language developed by Aptech Systems, inc. Data consistec 
of eight waves of achievement scores collected in the spring of each 
year from 1978 to 1985, and three ability scores collected in the 
fall of 1979 for a cohort of students as they progressed from Grade 1 
to 8. T^sts included the Prescriptive Reading Inventory, Diagnostic 
Mathematics Inventory, California Achievement Tests, and tests within 
the North Carolina Annual Testing Program. Results indicate that the 
use of growth curves as indices of change has numerous advantages 
over other methods, and that longitudinal data analyses are feasible 
for school districts when a suitable longitudinal database has been 
maintained. Growth curves focus on the use of multi-wave data, 
provide statistical advantages of multiple occasions of measurement, 
and emphasize the individual student. Nine tables and two graphs are 
included. (TJH) 



****************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



LONGIiJDINAL ANALYSES OF ACHfEVEMENT DATAFROM DURHAM 

COUNTY (NO SCHOOLS 



by 

Gary L, wniiamson 
Mark Appelbaum 
and 

Alex Epanchin 

' PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BV 



TO THE EDUCATfONAL RESOURCES 
INFORMATION CENTER (ERIC) " 



EDUCATIONAL flESOUTCES iNfOfiMATtON 
J CENTER (ERK;> 

if Tim 0OCorn«nt b«en rtP^Ot'uced a% 

Ongmttting it 
rr Minor Ct>ang«s riav« be«n madfr io I'rprove 



• P^tirs Dt vr«w Of Opinions »tal«d m tt>ii Oocu 
men! do flOt i^ccjs^niy reore^nl OffiCffli 



Paper presented at the annual meeting of the American Educational 
Research Association, April, 1988 New Orleans, Louisiana. 



Longitudinal Analyses of Acfiievement Data from 
Durfiam County (NO Scfiools 

Tfie purpose of tfiis study is twofold. First, it is intended as an 
illustration of what can be learned from the analysis of longitudinal 
data. The emphasis is on examining individual growth curves. Second, 
the study is intended to show one way of charting the progress of 
educational institutions, such as schools, and the effects of 
aggregation on the study of growth. 

This project was supported by a contract of the North Carolina 
State Department of Public Instruction to the L. L. Thurstone 
Psychometric Laboratory, University of North Carolina at Chapel Hill. 
Or. Mark Appelbaum secured the contract and made available the 
resources of the L. L. Thurstone Psychometric Laboratory for the 
execution of the study and the completion of this report. Computer 
programs, data analysis, and report composition were provided by the 
first author. 

Special thanks are due to the Durham County School System for 
supplying the data. Particular thanks are due Dr. Alex Epanchin, 
whose vision it was to create and maintain the longitudinal data base 
that makes this study possible. 

Background 

Recent Literature 

Literature on the measurement of change has traditionally dealt 
with two occasions (waves) of measurement. The difference score as 
a measure of change has thus received much of the attention in this 
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literature. However, because of overly restrictive assumptions (see 
Rogosa and Willett (1983b)) and the ]iiCk of a suitable statistical 
model for cliange(see Rogosa, Brandt and Zimowsl<i (1982)), many 
autfiors fiave condemned tfie difference score as a measure of cfiange 
and fiave recommended otfier approacfies (e.g.. Lord (1956, 1963), 
Tfiorndike ( 1 966), Cronbacfi and Furby ( 1 970), Nunnallij ( 1 973), and 
O'Connor (1972)). Baltes, Reese andNesselroade(J977) recognized 
tfie need for modeling change over time and the fact that perceived 
problems with traditional approaches to the measurement of change 
(e.g., low reliability of the difference score) were partly rt'ated to 
the fact that only two occasions had been used to study change. 

Recent developments in the measurement of change have 
clarified many of these issues and have moved a long way toward 
correcting previous misconceptions. These recent developments have 
generalized beyond two occasions of measurement to the situation 
where richer longitudinal data are available. 

Blomqvist (1977) used the straight-line growth model to 
investigate the relationship between change and initial status. 
Maximum likelihood estimates were provided for the covariance 
matrix of the parameters of a single measure. 

Rogosa, Brandt and Zimowski (1982) emphasized the moUsling of 
individual change and asserted that individual time paths are the 
"proper focus for the analysis of change." (p. 744) They stated a 
statistical model for the individual growth curve and exam'ned 
various assumptions in the measurement of change literature , They 
also investigated traditional measures of chartge (the difference 
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score, the improved difference score, the Lord-flcNeniar regression 
estimate, Bayesian growth curve estimates, and residual change 
measures) for two- wave aata. They discussed the statistical 
properties and reliability of each measure. Measures of change for 
multiwave data were also discussed and the advantages of multtwave 
data were noted. In their "Mottos for the Measurement of Individual 
Change", they summarized key points derived from a conceptual and 
mathematical framework for the measurement of individual change. 

Several other references provide new results for the 
measurement of individual change. Rogosa and Willstt (1983b) 
discussed the reliability of the difference score and pointed out 
oversights in previous literature. For example, previous 
investigations of the reliability of the difference score had (perhaps 
unknowingly) concentrated on cases where individual differences in 
true change do not exist, or exist only to a small degree. Rogosa and 
Willett illustrated that the difference score can be a reliable 
measure, and they showed how restrictive assumptions (e.g., equality 
of observed score variances at time 1 and time 2; equal reliabilities 
for each measure) have caused previous investigators to miss this 
fact. 

1 n more recent work, Wi 1 lett ( 1 985) and Rogosa and Wi 1 lett 
(1985) discussed correlates of change via models for systematic 
individual differences in growth. Their approach incorporates a model 
for growth and a model for individual differences in growth. A model 
with constant rate of growth and various models with nonconstant 
rates of growth were discussed. (Models with nonconstant rates of 
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growth include polynomial, linear state dependence, logistic and 
simplex models.) Rogosa, Floden and Vv tliett ( 1 984) and Rogosa and 
Willett (1983a) considered the use of tracking indices to assess the 
stability of individual differences over time. 

Rogosa, Willett and Williamson (1986) used many of the above 
techniques to analyze achievement scores on the Comprehensive Tests 
of Basic 3l<il!s (CTB3) for a cohort of approximately two hundred high 
school students. They also illustrated how some common indices 
from the profile similarity literature can be used to help examine 
growth on multiple measures. Finally, they introduced a prototypical 
student achievement report that highlights academic growth on 
multiple measures. 

Williamson (1986) investigated growth on multiple measures. 
Profiles of individual achievement and growth were investigated 
using using a straight-line growth model for individual longitudinal 
data sequences on several academic measures. A specific data 
analysis procedure was defined that provides ( 1 ) univariate and 
multivariate descriptions of achievement and growth, and (2) 
identification of intraindividual strengths and weal<nesses in 
achievement and rate of growth. 

Most recently, Bryk and Raudenbush ( 1 987) advanced the study of 
change by formulating a general model framework (hierarchical linear 
models) and using more sophisticated techniques to estimate the 
parameters of the models. (The pair of models used by Rogosa and 
Willett (1985) is a special case of the Bryk and Raudenbush approach.) 
Hierarchical linear models have been used t>y Bryk ( 1 987) and 



Raudenbush (1987) to study the effects of educational programs on 
student academic growth. 

The recent literature on the measurement of change focuses 
attention on models for individual growth. Questions about level and 
rate of learning on a single measure are phrased in terms of 
quantities derived from a specified model. This desirable property 
lends itself well to answering questions about growth since 
estimated parameters provide descriptions of individual performance 
and can be aggregated to provide descriptions of institutional 
performance. 

Mathematical Model 

A two-level hierarchcal linear model is used in this study. First, 

a model for individual growth is specified. Second, a model fs 

specified to relate the parameters of the first model to some 

relevant background variable, W, that is thought to be related to 

growth. Through this approach, relevant growth parameters are 

identified and their relationship to other variables is made explicit. 

The simplest type of individual growth is straight-line growth. 

In this case, each individual exhibits a constant rate of change. The 

mathematical model for straight-line growth for individual p is 

CpU) = Cp(0) ^ Op t (1) 

Where Cp(t) is the true score of individual p at time t and Op is the 
true rate of change of individual p. 
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There are numerous reasons for u^mg a straight-line growth 
model. They include parsimony, ease of Interpretation, frequent use 
by researchers, robustness (i.e., linear models often provide a good 
approximation even when growth is nonlinear), and the fact that 
linear models often fit empirical data well. 

There are of course limitations involved with the choice of such 
a simple model. Using a straight-line growth model, the researcher 
implicitly assumes that growth (however scaled) is constant in time, 
that the scale is continuous and extends ini initely in one or both 
directions (positive, negative). In addition, a straight-line model may 
be Inappropriate when certain kinds of measurement problems exist. 
For example, ceiling bounded effects are not modeled by a 
straight-line growth curve. 

When such problems are detected, alternative score scales or 
alternative models may be more appropriate. )n any case, fitting a 
straight-line growth model is a reasonable first step in investigating 
individual growth patterns. 

The second l^Jvel of the hierarchical linear model gives an 
explicit representation of the relationship of dp to some exogenous 
background variable, W (assumed to be constant over time). As noted 
by Rognsa and Willett (1985), "individual differences in growth exist 
when different individuals have different values of Op. Systematic 
individual differences Ingrowth exist when individual differences in 
a growth parameter such as dp can be linked with one or more Ws." 
(p. 205) A simple relationship is expressed by the model. 



ERIC 



6 

8 



(2) 



where \iq is the population mean of Q^; is the population mean of 
Wpi and ^ew population regression coefficient A nonzero 
value of ^0\^ indicates that W is a predictor of growth. The 
correlation, p0\^, is often more convenient, and is used in this report. 

The two-part model given in ( ! ) and (2) mal<es it possible to 
describe individual growth. In addition, it allows the investigation of 
systematic individual differences ingrowth. 

Computer Programs 

Most of the analyses presented in this study were produced by 
TinFPATH, a computer program for fitting individual straight-line 
growth curves to longitudinal data. The program was developed at 
Stanford University under the supervision and guidance of Professor 
David R. Rogosa. Ordinary least squares and maximum likelihood 
estimation are used to accomplish state-of-the art longitudinal data 
analyses. 

TIMEPATH is written in GAUSS, a matrix programming language 
developed by Aptech Systems, inc. (see Edlefson and Jones, 1984) for 
the IBM/PC and compatibles. TIMEPATH is used to fit individual 
str»ight-line growth curves, investigate the aptness of the 
straight-line model, and produce comprehensive statistical 
summaries relating to individual growth and systematic individual 
differences in growth. The program also produces an output file that 
can be used for subsequent analyses. 



The Statistical Analysis System for the PC (SAS/PC) was used to 
perform some descriptive amiyses. In addition, SAS/PC proved 
useful for preprocessing the data before using TIMEPATH, and for file 
management. 

Ait analyses were done on an IBM/PC AT using SAS/PC or GAUS5 
Given the general availability of both software and hardware, this 
study serves as a demonstration of the feasibility of studying 
achievement and qrowth using longitudinal data in a school system 
setting. 

Application 

The data were supplied by Durham County Schools. They consist 
of eight waves of achievement scores collected in the spring of each 
year from 1978 to 1985, and three ability scores collected in the fall 
of 1979 for a cohort of students as they progressed from grade one to 
grsJe eight. Special thanks go to Dr Alex Epanchin for supplying the 
data in a convenient form and for describing the nature of the data. 

The tests and forms are listed in Table 1. Appropriate levels of 
the Prescriptive Reading Inventory (PR!) and Diagnostic Mathematics 
Inventory (DM1) were administered to grades one and two in 
conjunction with the North Carolina Annual Testing Program. Various 
levels of the California Achievement Tests (CAT) were administered 
to grades three through eight in conjunction with the North Carolina 
Annual Testing Program and the local school district testing program. 
The PRl, DMl and CAT are produced by CTB/McGraw-Hill. 
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Table 1 



Achievement and Ability Tests 



Year Grade Test* Form Level 



Achievement Tests (Spring Testing) 



1978 


1 


PR! 




2 






uni 




Kea 


1Q7Q 


9 






A KcU 






OMt 




Green 


1980 


3 


CAT 


C 


13 


1981 


A 


CAT 


C 


14 


1982 


5 


CAT 


C 


15 


1983 


6 


CAT 


C 


16 


1984 


7 


CAT 


C 


17 


1985 


8 


CAT 


C 


18 






Ability Test (Fall Testing) 






1979 


3 


Cognitive Abilities Test 


1 


A 



PRI=Prescriptive Readina Inventory 
DMI-Diagnostic Mathematics Inventory 
CAT=Caltfornia Achievement Tests 
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Reading Total Scale Scores and Mathematics Total Scale Scores 
were chosen as the achievement measures to be analyzecl. Scale 
scores were chosen because chey reportediy have properties tfiat 
make tfiem suitable for longitudinal studies. Scale scores are a 
normal part of the reporting provided by CTB/McGraw-Hill for the CAT 
and were routinely reported at grades three through eight. The scale 
scores for grades one and two were estimated by CTB/McOraw-Hill 
from the PRI and DMI scores. (Analyses for Language Total and Total 
Battery are not included since the subtest (and consequently the total 
battery score) was unavailable for grades one and two.) 

Scores on the Cognitive Abilities Tests (Riverside Publishing 
Company) were collected at several times during the eight years. The 
scores on these tests during 1 979 were chosen as background 
exogenous variables for the illustration of analyses of systematic 
individual differences in growth. The theory used in the application 
of equation (2) requires that background variables be constant over 
time. While this is probably not true for ability test scores, they 
provide a ccrivenient illustration of the techniques, The substantive 
results found here regarding systematic individual differences in 
growth have meaning only to the extent th^jt :;ljility can be regarded 
as a constant attribute of th'* individual. 

From the data base, there were 667 observations that had 
student records for each of the eight years. Wien cases with missing 
d&ta were eliminated this reduced to 529 individuals with complete 
Reading Total records and 527 individuals with complete Mathematics 
T'^tal records. The individuals were spread among twelve schools in 
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the district. Table 2 shows the frequency of students from each 
school (Identified by an arbitrary two-digit school identification 
code) and the number of males and females in the total group. These 
two data files (one for reading, one for math) are the ones used for 
this study. 

Analyses Performed 

For each of the two data files^ preliminary descriptive analyses 
wer^ run to check the score distributions for obvious outliers or data 
errors. Frequency tables showing the number of individuals in each 
school and gender group were produced. Because of a limitation on 
the amount of space available to GAUSS on and IBM/PC AT, it was 
decided to analyze gender groups separately. Thus some statistics 
(for example, estimates of the reliability of the empirical rates) are 
reported only for gender groups and not for the total group, 

Tne TIMEPATH program was run for Reading Total Scale Score 
(male, female) and Mathematics Total Scale Score (male, female). An 
output file was created containing the estimated individual rates of 
change and other summary statistics for each analysis group (reading, 
males; reading, females; mathematics, males; mathematics^ females). 
The four output files were combined into two files (Reading, total 
group; Mathematics, total group) and SAS/PC was used to determine 
distribution statistics for the total group and for each school in the 
district. 

Finally effects of aggregation were investigated as follows 
First the mean growth rate and various ceritiles in the empirical 



ERIC 



1 1 



Table 2 



Sample Distribution for Reading and Mathematjcs 



School Reading Mathemati cs 

N N 

10 50 49 

n 43 43 

12 27 27 

13 53 51 

14 72 74 

15 31 31 

16 4 4 

17 25 26 

18 70 70 

19 57 58 

20 41 40 

21 56 54 
District 529 527 

Males 278 277 

Females 251 250 



12 

14 



cl)sr-ibut)ons of growth rates were calculated for the district and 
each school. These provide a sound picture of the nature of growth tn 
reading and mathematics at the district and school level. Next, the 
means and corresponding centtles of scale scores were calculated on 
each occasion of measurement for the district and for each school 
Then straight-line growth curves were fit to each of these 
"aggregated" scores (i.e., the scale score means, or centiles). The 
resulting growth rates, determined from fitting an "institutional 
growth model", were compared with mean (or centile) growth rates 
from the empirical distribution of individual growth rates. 

The procedure described in the previous paragraphs (particularly 
the application of the TlliEPATH program) produces a wealth of 
informati'' r about growth. The results described in the next '"ction 
illusti ate what is possible when suitable models for growth are used 
and longitudinal data are properly maintained. In addition, the 
analyses make it possible to sensibly chart institutional growth. 

Descriptive summaries of scale scores are presented first. Next 
the aptness of the straight-line model is investigated and results are 
summarized. Individual growth is discussed, statistical and 
psychometric characteristics are described, and group summaries are 
provided. Last, the effects of aggregation are discussed. 

Results 

Descriptive summary. An examination of the empirical 
distributions of scores revealed no peculiarities. This was expected 
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since the school district has likely done extensive data cleaning prior 
to this study 

Table 3 shows means and standard deviations for Reading Total 
and Mathematics Total scale scores at each g^ade, and for the ability 
(iQ) test scores in grade three The achievement test means show a 
steady increase from grade one to grade eight. There appears to be an 
iftcrease o\ about 35-40 scale score points per year on both reading 
and mathematics. The ability test scores indicate that the students 
were ulightly above average in verbal, mathematical and nonverbal 
ability near the beginning of their school career. 

Reporting achievement test score means as in Table 3 is the 
typical presentation made by educational institutions when trying to 
demorstrate growth. If each individual in the district is growing in 
straight-line fashion, then the trend in the means of Table 3 is 
actui'lly informative about the mean rate of growth in the district. 
However, the taMe does not make the rate of growth explicit. 
Furthermore, this presentation of the longitudinal data does not 
inform about the growth of individuals in the district, or about the 
variability m growth across individuals in the district. In the worst 
case, tt is possible that the growth function implied by the means is 
entirely different from the growth exhibited by individuals in the 
district. Finally, the table does not show an explicit connection 
between the growth implied by the achievement test means and the 
status in ability represented by the ability test means. 

It is much better to investigate individual growth first, before 
attempting to depict institutional performance. Then some of the 
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Table 3 

Means and Standard Deviations 



Grade 


Year 


Reading Cohort 
(N =329) 


Mathematics Cohort 
(N = 527) 






Mean 


S. D. 


Mean 










Achievement Scores 




I 


1978 


320.7 


31.9 


333.3 


25.9 


2 


1079 


383.6 


40.1 


385.0 


33.7 


3 


1980 


426.7 


51.0 


416.2 


32.7 


4 


1981 


470.2 


55.9 


454.6 


43.2 


5 


1982 




60.6 


493.7 




6 


1983 


536.0 


63.8 


525.7 


54.7 


7 


1984 


565.1 


67.9 


558.3 


63.5 


8 


1985 


593.6 


74.1 


588.9 


68.1 








Ability Scores (Fall i979) 




Verbal 




104.9 


14.3 


104.7 


14.4 


Mathematics 


10 a. 6 


15.2 


104.4 


15.2 


Nonverbal 


105.3 


14.3 


105.3 


14.2 
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above concerns can be addressed m a productive manner. The first 
step IS the choice of a suitable model for individual growth. The 
model represented by (1) and (2) is employed with the Durham County 
longitudinal data. The results follow. 

Adeouacu of the model . The model in (1) was fit to each 
individual's longitudinal data on Reading Total Scale Score. The same 
approach was taken with Mathematics Total Scale Score. Examining 
the individual values of the squared multiple correlation, R^, is one 
way of judging the adequacy of fit for each individual. Table 4 shows 
a seven-number summary of the distributions of R^ for reading and 
mathematics. 

Note that 95% of the individuals have R^ values greater than .85 
for reading and .86 for mathematics. The median values of R^ are .95 
and .96 for reading and mathematics, respectively! This seems to 
indicate that straight I ines are a remarkably good choice f or the 
individual growth model with this cohort of students. 

However, R^ can be misleading. There can be large values of R^ 
when the true growth model is actually nonlinear. Plots of the scores 
of selected individuals with largest values of and values of R^ 
near the median confirmed the linear trend. Plots of scores for 
individuals with lowest values of R^, or with high sums of squared 
residuals (S3RE3), are of course less clear. 

In order to quantify departures f rom I inearity, a quadratic 
component was tested for each individual fit. Only 33 individuals 
(6.3%) Showed a significant (« = .10) quadratic component for 
mathematics. Ninety-three individuals(17.6%) showed ssignificant 
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Table 4 

Seven-Number Summanes of r2 



*529 






*527 






median 




1 

.945 1 


medisn 


1 .961 




25% 


1 .915 


.964 1 


25% 


.938 


.977 


5% 


1 .853 


.985 1 


5% 


1 .863 


.990 


extreme 


1 .559 


.997 1 


extreme 


1 .451 


.996 



Reading Total Matfiematics Total 
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(oc = .10) quadratic component for reading. Plots of scores for 
selected students revealed that curvature may indeed be observed for 
a lew individuals. For other students, the curvature might be due to 
one stray point or to erratic scores. Usually, these cases were also 
Identified by a large value of 55RE5. 

While a number of individuals might better be modeled by a 
quadratic curve for their reading scores, it appears that the 
straight-line model is appropriate for the majority of individuals for 
both reading and mathematics. 

Figure 1 shows a histogram of the 55RE5 from each individual 
straight-line regression. These histograms show that relatively few 
individuals have large values of 55RE5. This is true of both reading 
and mathematics, and supports the conclusion that the straight-line 
model is appropriate. 

The ordinary least squares estimate of the rate of growth in the 
straight-line model ^s robust in the sense tha: even when the true 
model is quadratic^ the straignt-line slope provides an estimate of 
the average growth rate of the individual. This, along with the fact 
that most of the individuals seem to fit the straight-line model 
extremely well, implies that the straight-line model is suitable for 
these data aid should serve eminently well with these data as an 
Illustration of the growth curve approach to the measurement of 
change. 

Individual growth . The great advantage of fitting individual 
growth curves to longitudinal data is that growth rates can oe 
determined for each student and the quality of fit can be examined 
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individually. It is also frequently possible to identify aberrations in 
patterns of growth by examining the scores in conjunction with the 
model fit for an individual. This can lead to useful hypotheses about 
the observed behavior. For example, sometimes a poor fit can be due 
to one peculiar point. This can raise questions about the testing on 
that administration, or about the validity of the student's 
performance on that particular day. 

Abbreviated output appears in Appendix A showing fits to 
individual regressions for the first 56 male students in the reading 
analysis Students are identified by a numeric code. Then values of 
their ability scores are listed as through W3. These correspond to 
the Cognitive Abilities Test scores on Verbal, Mathematics and 
Nonverbal, respectively. Then each student's 0L5 growth rate is 
listed along with the associated value. The seventh column 
contains the individual increments to (labeled (i_rsq) due to the 
addition of a quadratic component to the straight-line model. The 
last eight columns show the scale scores from grades one through 
eight, labeled through Xg. Though not shown on the printout, 55RE5 
and other diagnostics were computed for each individual. 

f^y examining distributions of the growth rates, R^, 55RE5 or 
other quantities, it is possible to identify indiviooals with singular 
characteristics. For example, it is possible to determine that 
individual 77431 1 has one of the poorest fits among male students. 
It appears that the quadratic component is significant. A glance at 
the scores would suggest that this is largely due to the eighth grade 
score, which shows a decided drop from the previous scores. Another 
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student, 734810, has one of the fastest growth rates (55.8) among 
males. The value of is high for this individual, and a glance at the 
scores conf irms that the estimated growth rate is an accurate 
indication of fast and steady growth. 

Table 5 shows five-number summaries of the growth rates on 
reading and mathematics for the total group of students and for males 
and f emales separately. For the total group, it is apparent that the 
median growth rates are nearly the same on reading and math, but 
that there is slightly more variability in mathematics growth, i.e., 
the extreme growth rates are more extreme for mathematics. This is 
Lrue among males and among f emales as well. Females and males 
show very similar distributions for both reading and mathematics. 

Statistical and Psychometric characteristics. Because of the 
nature of the models (1) and (2), it is possible to estimate several 
important quantities. One key quantity is the variance of the true 
rates of growth in the population, C^. Once this estimate is 
available, it is possible to estimate the reliability of the estimated 
rate of change, 6, and the standard error of 6. These quantities are 
shown in Table 6. Separate estiamates are provided for reading and 
mathematics by gender group, (it would be preferable to provide 
these for the total group, but space limitations with GAUSS on the 
IBM/PC make this more difficult.) The estimated variances of the 
true rates of change range from 40.8 to 65.9. The reliability of § 
ranges from .731 to .858. (Recall that much traditional literature 
focused on the low reliability of change scores!) The reliability is 




*529 

median 

25% 

extreme 



*278 

median 

25% 

extreme 



Table 5 

Five-Number Summaries of Rate 

Total Group 

*527 

median 



373 
32.5 43.1 
16.3 597 
Reading 



36.8 
32.6 41.5 
1 6 3 59.7 
Reading 



25% 

extreme 



Females 



*277 

median 

25% 

extreme 



36.2 

30.7 41.0 
9.7 64.2 
Mathematics 



36.4 
31.5 41.1 
9.7 64.2 
Mathematics 



Males 



*25I 
median 

25% 

extreme 



37.7 
32.1 44.4 
17.3 59.0 
Reading 



*250 

median 

25% 

extreme 



35.6 
29.7 41.2 
11.6 62.2 
Mathematics 
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Table 6 



Estimates of Gtatistica) and Psychometric Quantities 



Quantity Reading Mathematics 

Estimated 

Females Males Females Males 





40.812 


58.056 


46.028 


65.883 


Reliability of § 


.731 


.755 


.827 


.858 


Standard Error of § 


3.872 


4.341 


3.0Q9 


3.301 


Tracking index, "6 


.757 


.742 


.721 


.687 


Correlation between 
e and initial status 


.708 


.692 


.653 


.483 


Correlation between 
eand Verbal ability 


.681 


646 


.686 


.614 


Correlation between 
e and Mathematics 
ability 


.557 


.627 


.700 


.666 


Correlation between 
0 and Nonverbal 
ability 


.560 


546 


598 


.534 



23 



somewhat higher to" mathematics than for reading, but it is very 
respectable for both subjects Standard errors are also provided. 

Another interesting quantity listed in Table 6 is an estimate of 
the tracl<ing index of Foulkes and Davis (1981). Denoted this 
tracking index reflects the consistency of individual differences in 
growth across individuals. Alternatively, the index indicates the 
degree to which individual growth curves are parallel (or "tracking" 
each other). The index ranges between zero and one, with higher 
values indicating a greater degree of tracking. Gamma is the 
empirical probabiMty that two randomly chosen growth curves do not 
cross in the observed range of time Values of in excess of .5 are 
taken to indicate tracking. 

As can be seen from Table 6, tracking occurs for both males and 
females on both reading and mathematics. This c?n be interpreted to 
mean that there are relatively few crossings of growth curves within 
each group. An alternative interpretation is that academic ranking 
within each group is being maintained across time. 

The )ast four rows of Table 6 relate to the investigation of 
systematic individual differences m growth. Initial status is often 
taken to be an important correlate of rate of growth. For these data, 
initial status is taken to be the achievement level at grade one. There 
are three other candidate predictors as well. They are the three 
ability scores from the Cognitive Abilities Test-- Verbal, 
Mathematics, and Nonverbal scores. The last four rows of Table 6 
show the estimated correlations between rate of change and the four 
potential predictors of change mentioned above. 
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The correlation between change and initial status is moderate 
(.48) for males on vhe mathematics test. However, for females it is 
.65 The correlation between change and initial status ts higher on 
the reading test— .69 among males and .71 among females 

The correlations between rate of change and the ability test 
scores range from .534 (the correlation among males between rate of 
change in mathematics achievement and Nonverbal ability in grade 
three) and .700 (the correlation among fem?les between the rate of 
change in mathematics achievement and Mathematics ability in grade 
three). Thus the ability scores are reasonably good predictors of rate 
of growth in these data. 

Summarizing schooLgrowth. By examining the empirical 
distributions of the estimated rates of change within each school, it 
is possible to produce an informative picture of the growth in each 
school in the district. Table 7 shows the mean growth rate in each 
school and the district. In addition, Table 7 shows the five number 
summary of growth rates in each school. It is apparent that School 
20 has the highest average growth rate on Reading Total Scale Scores 
and School 16 has the lowest. There are however subst.jntiai 
differences in the number of students in each school. For example. 
School 16 has only four students in this cohort. Nevertheless, all the 
schools demonstrate positive average growth. 

The five-nLffnber summaries are more informative. Scanning the 
minimum values of the rate of growth, it is apparent that all students 
are showing progress on both Reading Total and Mathematics Total. 
The smallest individual growth rate is 9.7 in School 18 for 




T3ble 7 



Means and Selected Quanttles of -he Empinca! Rate Distribution for 

each School and t'y- District 



School 


N 


Mean 


Minimum 


25% 


Median 


75% 


Maximum 








Reading 








10 


50 


36.1 


26.0 


31.3 


34.6 


39.9 


55.8 


n 


43 


35.5 


18.0 


30.9 


35.8 


40.5 


50.1 


12 


27 


36.6 


19.2 


29.3 


36.4 


40.7 


58.8 


1 T 
1 J 






21.4 


30.8 






7 


14 


72 


37.6 


17.3 


31.5 


38.6 


42.4 


56.5 


15 


31 


42.6 


32.5 


38.2 


43.2 


45.5 


55.7 


16 


4 


23.5 


16.3 


19.1 


24.0 


27.9 


29.9 


17 


25 


38.6 


25.0 


33.2 


37.0 


42.6 


56.1 


18 


70 


36.1 


20.7 


31.2 


35.1 


39.7 


58.4 


19 


57 


36.2 


19.3 


31.3 


36.4 


40.1 


53.6 


20 


41 


43.3 


28.7 


37.3 


42.8 


50.5 


57.7 


21 


56 


40.4 


20.7 


34.9 


38.6 


46.1 


59.0 


Total 


529 


37.9 


16.3 


32.5 


37.3 


43.1 


59.7 








Mathematics 








10 


49 


32.5 


11.6 


27.7 


33.6 


36.7 


56.2 


1 1 


43 


347 


18.5 


29.6 


35.4 


39.5 


49.6 


12 


27 


33.4 


18.8 


25.8 


35.0 


40.2 


50.5 


13 


51 


37.5 


19.6 


31.4 


37.0 


42.2 


56.3 


14 


74 


37.0 


12.9 


33.3 


37.2 


42.6 


53.8 


15 


31 


41.1 


32.2 


36.9 


39.0 


44.1 


642 


16 


4 


23.2 


14.2 


18.1 


22.2 


28.4 


34.3 


17 


26 


36.7 


25.6 


31.0 


35.4 


42.9 


55.8 


18 


70 


33.4 


9.7 


27.9 


33.5 


38.9 


49.6 


19 


58 


33.8 


15.2 


28.6 


34.2 


38.3 


544 


20 


40 


39.6 


22.2 


34.5 


39.1 


44.4 


62.2 


21 


54 


39.4 


21.3 


34.1 


37.8 


46.6 


56.9 


Total 


527 


36.0 


9.7 


30.7 


36.2 


41.0 


64.2 
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mathematics. By using the individual fits, this individual could be 
identified and perhaps hypotheses could be offered about the 
relatively low performance. On the opposite end of the spectrum, the 
highest individual growth rate is 642 m School 1 5, and again, it ts on 
mathematics. 

By scanning the five-number summaries for each school, It is 
apparent (as it was from the mean growth rates) that there is 
variability across schools in the central tendency of the distributions 
of growth or learning, tn addition, there is some variability in the 
range of growth within a school. One school shows a range of more 
than forty scale score points (i.e.. School 18, mathematics,/ while 
another has a range of less than 30 scale score points (i.e., School 20, 
reading). Stiit, all schools seem to be characterized by consistently 
strong growth patterns. This is of course reflected in the District 
summaries for reading and mathematics. 

Effects of agarenation . When polynomial growth curves are used, 
the collection of individual growth curves has the property that the 
mean of the individual growth rates is equal to the growth rate of the 
mean scores. This property has been called dynamic cons fstency. 
However, this property does not necessarily characterize other types 
of growth curves, nor does it necessarily apply when other types of 
summary statistics are applied (e.g., the median, or other quantiles). 
The purpose of this section is to empirically investigate the effects 
of using the quantiles of the score distributions at each occasion 
(rather than the quantiles of the distribution of individual growth 
rates) as summaries of longitudinal performance. 
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Table 8 Shows the growth rates determined from fitting a 
straight-Hne model to various summary quantities. Six different 
summary statistics are used They are the mean, the minimum, the 
25^^ quantite, the median, the 75^^ quantile, and the maximum These 
growth rates should be compared to the growth rates in Table 7 to 
determine how closely growth rates based on summary statistics 
reflect the actual empirical distribution of growth. 

Note first of all that the column of means in Table 8 is exactly 
the same as the column of means in Table 7. This is true for botn 
reading and mathematics. This reflects the dynamic consistency of 
polynomial growth curves mentioned above. 

Next note the column of "median" growth rates in Table 8 and 
compare this with the actual median growth rates in Table 7. The 
two sets of "medians" are not equal. The values for each school in 
Table 8 are sometimes the same as the corresponding values in Table 
7; but more often, they are different. There is no discernible pattern. 
Table 8 gives higher values for some schools and lower values for 
other schools. 

Comparing the "minimum" ai.,1 "25^^ quantile" values in Table 8 
with those in Table 7, it is notable that the values in Table 8 are 
most often larger than those in Table 7. Generally it appears that 
fitting growth curves to the minimum values of the scale scores (or 
the 25*^^ quantiles) yields an overestimate of the minimum (or 25*^^ 
quantile) of the empirical distributions of the growth rates. Th*s is 
true regardless of whether the level of "aggregation" is at the school 
or district level. 
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Table 8 

Rates Estimated bu Fitting a Straight-Line Growth Model to the Means 
or Various Quantiles Within Each School and the District 



School Mean Minimum 25% Median 75% Maximum 



Reading 



10 


36.1 


30.3 


342 


36.0 


37.2 


52.2 


n 


35.5 


21.0 


31.2 


35.4 


39.6 


51.7 


12 


36.6 


23.2 


28.9 


36.5 


41.3 


52.8 


13 


37.7 


19.1 


32.5 


37.2 


40.9 


55.2 


14 


37.6 


19.5 


31.9 


38.7 


41.5 


52.6 


15 


42.6 


36.2 


40.3 


40.9 


43.9 


52.2 


16 


23.5 


16.3 


19.3 


23.2 


27.7 


31.3 


17 


38.6 


30.9 


33.5 


36.6 


41.7 


55.3 


18 


36.1 


30.0 


33.2 


36.4 


38.9 


50.4 


19 


36.2 


25,4 


36.3 


35.6 


35.8 


52.5 








39.1 


42.1 


Hi A 


DD. / 


21 


40.4 


29.6 


37.5 


41.4 


41.7 


53.7 


District 


37.9 


23.3 


342 


37.6 


39.6 


58.1 








Mathematics 






10 


32.5 


16.4 


30.0 


34.2 


34.6 


50.8 


! 1 


34.7 


25.9 


31.2 


35.5 


37.7 


48.2 


12 


33.4 


21.0 


25.4 


35.0 


39.4 


50.0 


13 


37.5 


28.6 


33.0 


37.0 


41.2 


53.6 


14 


37.0 


19.3 


35.8 


38.2 


41.6 


47.6 


15 


41.1 


35.6 


37.2 


40.6 


43.3 


51.9 


16 


23.2 


20.1 


20.3 


21.1 


26.1 


30.6 


17 


36.7 


27.8 


323 


347 


41.2 


56.5 


18 


33.4 


19.5 


31.4 


34.2 


36.4 


47.3 


19 


33.8 


17.6 


29.3 


346 


38.1 


52.5 


20 


39.6 


24.6 


37.5 


39.7 


41.9 


53.9 


21 


39.4 


22.1 


35.1 


36.2 


43.3 


51.3 


District 


36.0 


16.i 


32.6 


36.5 


39.8 


56.5 
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Finally, compare the "75^^ quantile" and "maximum" columns in 
Table 8 with the corresponding columns in Table 7. Again, there are a 
few exceptions, but in general, the numbers in Table 8 are smaller 
than those in Table 7. Determining growth from the upper quantlles 
of the school or district scale score distributions appears to give an 
underestimate of the empirical growth rates at corresponding points 
in the school or district distributions of growth. 

Concluding Remarks 

Recent literature in the measurement of change focuses on the 
use of growth curves for modeling individual growth. This approach 
has numerous advantages over previous methods. Foremost, it 
focuses attention on the use of multi-wave data for determining 
characteristics of growth among individuals. There are statistical 
rewards for using multiple occasions of measurement. The rewards 
include improved estimation, increased precision and better 
reliability Just as important, however, the use of an individual 
growth model focuses attention on individual students and makes 
them the first level of analysis. Understanding institutional growth 
is secondary to understanding individual growth. 

The results presented in this report show that longitudinal data 
analyses are feasible for school districts when a suitable 
longitudinal diita base has been maintained. The payoff for 
maintaining such data bases is a wealth of information about 
individual growth that has not been available to school personnel 
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before This includes estimates of individual growth; assessments of 
the quality of the growth model for each individual, diagnostic 
mforrr jtion that may help teachers better understand the 
performance of each student; and, information about correlates of 
learning. In addition, new ways of describing institutional 
performance over time become available. These include estimates of 
"typical" growth and descriptions of the distribution of growth in an 
institution. 

Falling to focus first on individual growth has it;, costs. 
Examining the mean level of performance in schools will give an 
accurate assessment of the average growth rate when polynomial 
growth models are used. However, such analyses give very limited 
information about growth in the schools, and their accuracy is not 
guaranteed when other (non-polynomial) growth models are used. 
Finally, attempting to determine institutional growth characteristics 
from auantiles of the scale score distributions can be misleading and 
seems to introduce systematic error at the extremes of the score 
distributions. 
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114 


181 


tlf 


32.5 


17>4 


«,5 


3^3 


376 


445 


46^ 


513 




56^ 


5^ 


71^8 Ef 


fl 


n 


1*2 


31.3 


16-7 


L3 
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373 


43B 


456 


4»4 


4lt 
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78 


8S 


IfS 


52J 


18.1 


f.2 


3L2 


331 


378 


448 


416 


535 
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41 >7 


13.6 
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4t8 


493 


531 


56i 


at: 


c^^ 


7mii 


n 


17 


IH 


3*.8 


1S.I 


f>* 


355 


361 


398 


412 


499 


m 


516 


5S8 


71S91I 


m 


118 


in 


32.7 


88J 


6.2 


344 


43S 


465 


58f 


548 


53^ 


6P6 


57C 


72^911 


114 


m 




36>3 


13>6 


4,1 


332 


3S6 


4S4 


438 


5f8 


531 


537 


59S 


mii^ 


93 






36J 


18 >2 


f>4 


324 


379 


392 


449 


476 


535 


548 


574 


7273il 


\4h 


M 


lf6 


42.8 


«,7 


f>l 


328 


385 


423 


45J 


5*4 


57: 


6*0 


c23 


7c Ten 


Iff 


lift 


1*8 


37.7 


17-5 


8>5 


341 


365 


413 


451 


499 


557 


541 


5«E 


n^ttii 


m 


113 


123 


5S>8 


16>8 


2.# 


335 


441 


412 


558 


631 


64^ 


627 


75: 


TlStlf 


ilk 


131 


135 


3S.7 


14 >3 


2.1 


348 


414 


479 


475 


518 


557 


-.11 


595 


7Jt1ll 




H 


184 


4SJ 


18 >5 


t>9 


3^3 


3i# 


4E3 


469 


584 


541 


5a? 


923 


737S1I 


w 


17 


112 


29. 1 


85 >8 


13.2 r 


211 


7!2 


398 


;56 


456 


481 


493 


iC8 


738311 




tfl 


1* 


36J 


94-6 


2*3 


326 


396 


423 


488 


5*8 


545 


:55 


5^ 


741311 






til 


34 >l 


97 >e 


l>t 


289 


343 


347 


444 


439 


459 


517 


521 


7^ftlf 


111 


44 


\iz 


37-7 


14>8 


2-1 


216 


385 


437 


466 


481 


555 


542 


y>i 


7oeii 


*5 


17 


1*5 


27.7 


1L5 


4>3 


335 


352 


423 


441 


459 


5*8 


41tt 


5cJ 


74B3II 




n 


«t 


32.5 


12.3 


L4 


383 


341 


383 


440 


476 


485 


4^ 


558 


75S«1I 


Itc 




17 


31,3 


15.9 


L2 


372 


363 


411 


443 


476 


54* 


577 


6!* 


754311 




14 


73 


e8>4 


78.9 


l#.l 


»3 


4H 


437 


414 


471 


54^ 


5:8 


ii; 


"f^y* I 




IZS 


I2S 


34J. 


84.3 


5.3 ^ 


338 


414 


581 


543 


5*4 


541 






757911 


117 


Iff 


125 


27.1 


11.4 


f.2 


364 


448 


478 


4*1 


4*1 


512 


55» 


59! 


7aietl 


n 


B7 


84 


3*.1 


14.3 


1.1 


315 


382 


381 


438 


^1 


416 


5** 


545 


741911 




tn 


U4 


45.8 


1L4 


2-5 


359 


367 


445 


553 


539 


633 


612 


«56 


74ntt 


\H 


111 


tl3 


33.8 


1S-1 


2.3 


3;2 


4** 


154 


475 


•B1 


54^ 


56E 


583 


7*4311 


1*7 


m 


111 


42.8 


17-9 


II 


364 


374 


423 


472 


5*4 


557 


51* 


*t8 


7^7811 


« 


93 


91 


21>S 


9*a 


4.1 


216 


351 


315 


445 


473 


444 


in 


,523 


ti^ll 


f8 


1(3 


13 


3l>4 


15.3 


2*6 


313 


341 


429 


461 


4V6 


571 


531 


58> 


771311 


131 


tis 


131 


43*5 


11.5 


1.6 


341 


421 


492 


483 


531 


633 


5ai 


ric 


771811 


il7 


111 


1*2 


36.7 


If .8 




335 


4|4 


478 


451 


525 


551 


clI 


5r 


773BU 


tl4 


Iff 


lf7 


37.3 


73.1 


21.1 > 


335 


4*1 


511 


5i8 


«39 


488 


6** 


5&7 


774311 


14 


w 


11 


2*,t 


73.7 


5-7 * 


316 


361 


315 


3*1 


4!2 


45A 


515 


*?t 


774811 


131 


116 


na 


37-7 


ll.f 


!.4 


372 


4** 


511 


518 


53^ 




d*7 


j27 


77681 i 




97 


Iff 


S5>4 


9L5 




341 


336 


381 


4*4 


U2 




517 


4~^ 


mm 


Icl 


;3f 


tS3 


S*.l 


92 > 




555 


;55 


4'2 


456 


5*8 


666 




714 


nm\ 


Ifl 


m 


17 


2*>8 


87.4 


If.f 


341 


369 


429 


«51 


462 


4:^ 


438 


4?E 


791911 


13E 


131 


J3S 


+4-1 


95.5 


2-9 




455 


h78 


553 


M 


b^f 




43e 


793811 


lU 


1?7 


122 


43>3 


12.1 


5>* > 


3^8 


335 


4'f2 


5J9 




587 


^19 


6io 


78S8II 


m 


1!7 


19 


33.1 


V6-4 




355 


415 


4o5 


^66 


512 


521 


5:5 


C13 


7873U 


VA 


121 


1*5 


3!>8 


11.7 




372 


312 


492 


4c3 


531 


557 


5*2 


59S 


T8S91S 


74 


75 


61 


11.2 


82 J 




313 


211 


347 


378 


^*4 


*f4 


i58 


*r 


79f3lS 


91 


U7 


117 


41.5 


99.1 




3^5 


376 


336 


458 


412 


535 


555 


S'5 


irsBU 


11 


82 


ife 


3IJ 


12.5 


5*8 


254 


136 


375 


4fl 


426 


4i7 


*t< 


4^ 




lis 


lf4 


tE3 


33.8 


85.9 




316 


361 


481 


451 


4o5 


W 


*S2 


5^5 


7mia 


111 


?S 


IE 


48.7 


96>6 




»5 




418 


*c7 


*61 


VJ 




57i 






Iff 


lf3 


37.6 


1**2 


;>7 ♦ 


33* 


410 


4*9 


521 


513 


57t 


577 


5*1 


7f89lc 


H 


94 


113 


S6.1 


11*5 




287 


325 


3S5 


411 


373 


458 


-27 


5H 


7tKta 


n 


83 


12 


24.« 


13.5 




318 


3W 


H2 


414 


*41 


431 


584 


4^1 


mm 


at 


97 


11 


5S.* 


16.2 




293 


32c 


324 


378 


484 


435 


•7! 


44»4 




86 


84 


IH 


ai.3 


^5*7 




313 


J56 


379 


431 


451 


ktf 


512 


5^1 


81131S 




IIS 


1*6 


52.4 


11.6 




32b 


4«3 


454 


577 


63V 


M 


6Tc 


73J 


032B13 


81 


1*7 


117 


35*2 


8»>1 


t4J > 


299 


425 


4^5 


521 


529 


543 


551 


57^ 


831313 


111 


tf7 


1*1 


38.4 


I5>f 




338 


374 


445 


*58 


531 


564 


5r? 


511 


83^8] 3 




95 


17 


34>2 


98 >9 


7.2 f 


211 


324 


:£9 




461 


*91 






84^3!: 


m 


E19 


na 


44>a 


99.7 


9,^ *^ 


:i3 


335 


4&5 


543 


5S4 


cc- 

•V * 






944^13 


lis 


119 


115 


52J 


96*3 


M 


322 


411 




433 


571 








845313 


71 


sr 


98 


2K4 


55,9 


L9 


313 


.3?* 


a4S 


117 


It? 


479 




«l£ 



33 



REFERENCES 



Baltes, P. B., Reese, K W., & Nesselroade, J. R ( 1 977) Life-span 
develoDmental osUcholoQU: infrofiuction to research methods . 
Monterey; BrooksZCole Publishing Company. 

Biomqvist, N. ( 1 977), On the relation between change and initial 
value. Journal of the American Statistical Association. 72(360), 
746-749 

Bryk, A. S. ( 1 987). A paradigm for assessing the effects of 
educational programs on student growth, paper presented ai the 
annual meeting of the American Educational Research Association, 
Washington, D.c. 

Bryk, A. S., & Raudenbush, S, W. (1987). Application of hierarchical 
linear models to assessing change. PsucholoQical Bulletin. 101(1). 
147-156. 

Cronbach, L. J., & Furby, L. ( 1 970). How we should measure 
"change"~or should we? P^cholaolcal Bulletin. 74. 68-80 

Lord, F. M. ( 1 956). The measurement of growth. Ed ucational and 
PsucholoQical Measurement. \6, 42 1 -457.. 

Lord, F,M, (1963). Elementary models for measuring change. InC. W. 
Harris (Ed.). Problems in measuring change (dp. 21-38). rladison: 
The University of Wisconsin Press. 

Nunnally.. J. C, ( 1 973). Research strategies and measurement methods 
for investigating human development, in J. R. Nesselroade & H. W. 
Reese (Eds.l Life-soan developmental psuchologu: Methodologic al 
issues (pp. 87-109). New York: Academic Press." 

O'Connor, E. F., Jr. (1972). Extending classical test theory to the 
measurement of change.. Review of Educational fiesearcn. 42. 73-97, 

Raudenbush, S. (1987). Applications of hierarchical linear models to 
the study of educational programs: A review. Paper presented at 
the annual meeting of the American Educational Research 
Association, Washington, D. C. 

Rogosa, D. R., Brandt, D., & Zimowskl, M. (1982). A growth curve 
approach to the measurement of change. PsucholoQi cal Bulletin. 92, 
726-748. ~^ ' 

Rogosa, D. R., Floden, R., & Willett, J. B. (1984). Assessing the 
stability or teacher behavior. Journal of Educational Psychology. 
76, 1000-1027. 



ERIC 



34 :is 



Rogosa, D. R., & Willett, J. B. (1983a). Comparing two indices of 
tracking (Letter to the editor! Biometrics. 39. 795-796 

RoQosa, 0. R., & Willett, J. B. (1983b). Demonstrating the reliability 
or the difference score in tfie measurement of cfiange. Journal of 
Educational Measurement. 20, 335-343. 

Roqosa, D. R., & Willett, J B. (1985). Understanding correlates of 
cnange by modeling Individual differences in growth 
Psgchometrika. ^ 203-228. 

Rogosa, D. R., Willett, J. B., & Williamson, G. L (1986). Tools for the 
study of school learning: Implementation of statistical methods for 
the analysis of student growth in achievement. Unpublished 
manuscript. 

Thorndike, R. L. ( 1966). Intellectual status and intellectual growth. 
Journal of E^ lucational Psu rhologu. 57. 121-127. 

Willett, J. B. (1985). Investigating systematic individual differences 
in academic growth. Unpublished doctoral dissertation. Stanford 
University. 

Williamson, G. L. (1986). AssessinCLAcademic Growth on Multiple 
Measures. Unpublished doctoral dissertation, Stanford University. 



ERIC 



35 :\7 



